Lightness constancy in reality, in virtual reality, and on flat-panel displays.

Virtual reality (VR) displays are being used in an increasingly wide range of applications. However, previous work shows that viewers often perceive scene properties very differently in real and virtual environments and so realistic perception of virtual stimuli should always be a carefully tested conclusion, not an assumption. One important property for realistic scene perception is surface color. To evaluate how well virtual platforms support realistic perception of achromatic surface color, we assessed lightness constancy in a physical apparatus with real lights and surfaces, in a commercial VR headset, and on a traditional flat-panel display. We found that lightness constancy was good in all three environments, though significantly better in the real environment than on the flat-panel display. We also found that variability across observers was significantly greater in VR and on the flat-panel display than in the physical environment. We conclude that these discrepancies should be taken into account in applications where realistic perception is critical but also that in many cases VR can be used as a flexible alternative to flat-panel displays and a reasonable proxy for real environments.

Perceiving depth and motion in depth from successive occlusion.

Occlusion, or interposition, is one of the strongest and best-known pictorial cues to depth. Furthermore, the successive occlusions of previous objects by newly presented objects produces an impression of increasing depth. Although the perceived motion associated with this illusion has been studied, the depth percept has not. To investigate, participants were presented with two piles of disks with one always static and the other either a static pile or a stacking pile where a new disk was added every 200 ms. We found static piles with equal number of disks appeared equal in height. In contrast, the successive presentation of disks in the stacking condition appeared to enhance the perceived height of the stack-fewer disks were needed to match the static pile. Surprisingly, participants were also more precise when comparing stacking versus static piles of disks. Reversing the stacking by removing rather than adding disks reversed the bias and degraded precision. In follow-up experiments, we used nonoverlapping static and dynamic configurations to show that the effects are not due to simple differences in perceived numerosity. In sum, our results show that successive occlusions generate a greater sense of height than occlusion alone, and we posit that dynamic occlusion may be an underappreciated source of depth information.

Familiar size affects perception differently in virtual reality and the real world.

The promise of virtual reality (VR) as a tool for perceptual and cognitive research rests on the assumption that perception in virtual environments generalizes to the real world. Here, we conducted two experiments to compare size and distance perception between VR and physical reality (Maltz et al. 2021 J. Vis. 21, 1-18). In experiment 1, we used VR to present dice and Rubik's cubes at their typical sizes or reversed sizes at distances that maintained a constant visual angle. After viewing the stimuli binocularly (to provide vergence and disparity information) or monocularly, participants manually estimated perceived size and distance. Unlike physical reality, where participants relied less on familiar size and more on presented size during binocular versus monocular viewing, in VR participants relied heavily on familiar size regardless of the availability of binocular cues. In experiment 2, we demonstrated that the effects in VR generalized to other stimuli and to a higher quality VR headset. These results suggest that the use of binocular cues and familiar size differs substantially between virtual and physical reality. A deeper understanding of perceptual differences is necessary before assuming that research outcomes from VR will generalize to the real world. This article is part of a discussion meeting issue 'New approaches to 3D vision'.

The impacts of lens and stereo camera separation on perceived slant in Virtual Reality head-mounted displays.

Stereoscopic AR and VR headsets have displays and lenses that are either fixed or adjustable to match a limited range of user inter-pupillary distances (IPDs). Projective geometry predicts a misperception of depth when either the displays or virtual cameras used to render images are misaligned with the eyes. However, misalignment between the eyes and lenses might also affect binocular convergence, which could further distort perceived depth. This possibility has been largely ignored in previous studies. Here, we evaluated this phenomenon in a VR headset in which the inter-lens and inter-axial camera separations are coupled and adjustable. In a baseline condition, both were matched to observers' IPDs. In two other conditions, the inter-lens and inter-axial camera separations were set to the maximum and minimum allowed by the headset. In each condition, observers were instructed to adjust a fold created by two intersecting, textured surfaces until it appeared to have an angle of 90°. The task was performed at three randomly interleaved viewing distances, monocularly and binocularly. In the monocular condition, observers underestimated the fold angle and there was no effect of viewing distance on their settings. In the binocular conditions, we found that when the lens and camera separation were less than the viewer's IPD, they exhibited compression of perceived slant relative to baseline. The reverse pattern was seen when the lens and camera separation were larger than the viewer's IPD. These results were well explained by a geometric model that considers shifts in convergence due to lens and display misalignment with the eyes, as well as the relative contribution of monocular cues.

Shape judgments in natural scenes: Convexity biases versus stereopsis.

Determining the relief of upcoming terrain is critical to locomotion over rough or uneven ground. Given the significant contribution of stereopsis to perceived surface shape, it should play a crucial role in determining the shape of ground surfaces. The aim of this series of experiments was to evaluate the relative contribution of monocular and binocular depth cues to judgments of ground relief. To accomplish this goal, we simulated a depth discrimination task using naturalistic imagery. Stimuli consisted of a stereoscopically rendered grassy terrain with a central mound or a dip with varying height. We measured thresholds for discrimination of the direction of the depth offset. To determine the relationship between relief discrimination and measures of stereopsis, we used two stereoacuity tasks performed under the same viewing conditions. To assess the impact of ambiguous two-dimensional shading cues on depth judgments in our terrain task, we manipulated the intensity of the shading (low and high). Our results show that observers reliably discriminated ground reliefs as small as 20 cm at a viewing distance of 9.1 m. As the shading was intensified, a large proportion of observers (30%) exhibited a strong convexity bias, even when stereopsis indicated a concave depression. This finding suggests that there are significant individual differences in the reliance on assumptions of surface curvature that must be considered in experimental conditions. In impoverished viewing environments with limiting depth cues, these convexity biases could persist in judgments of ground relief, especially when shading cues are highly salient.

Stereoscopic depth constancy for physical objects and their virtual counterparts.

Stereopsis plays an important role in depth perception; if so, disparity-defined depth should not vary with distance. However, studies of stereoscopic depth constancy often report systematic distortions in depth judgments over distance, particularly for virtual stimuli. Our aim was to understand how depth estimation is impacted by viewing distance and display-based cue conflicts by replicating physical objects in virtual counterparts. To this end, we measured perceived depth using virtual textured half-cylinders and identical three-dimensional (3D) printed versions at two viewing distances under monocular and binocular conditions. Virtual stimuli were viewed using a mirror stereoscope and an Oculus Rift head-mounted display (HMD), while physical stimuli were viewed in a controlled test environment. Depth judgments were similar in both virtual apparatuses, which suggests that variations in the viewing geometry and optics of the HMD have little impact on perceived depth. When viewing physical stimuli binocularly, judgments were accurate and exhibited stereoscopic depth constancy. However, in all cases, depth was underestimated for virtual stimuli and failed to achieve depth constancy. It is clear that depth constancy is only complete for cue-rich physical stimuli and that the failure of constancy in virtual stimuli is due to the presence of the vergence-accommodation conflict. Further, our post hoc analysis revealed that prior experience with virtual and physical environments had a strong effect on depth judgments. That is, performance in virtual environments was enhanced by limited exposure to a related task using physical objects.

Perceived depth modulates perceptual resolution.

Humans constantly use depth information to support perceptual decisions about object size and location in space, as well as planning and executing actions. It was recently reported that perceived depth modulates perceptual performance even when depth information is not relevant to the task, with faster shape discrimination for objects perceived as being close to the observer. However, it is yet to be determined if the observed "close advantage" reflects differences in psychophysical sensitivity or response bias. Moreover, it is unclear whether this advantage is generalizable to other viewing situations and tasks. To address these outstanding issues, we evaluated whether visual resolution is modulated by perceived depth defined by 2D pictorial cues. In a series of experiments, we used the method of constant stimuli to measure the precision of perceptual judgements for stimuli positioned at close, far, and flat perceived distances. In Experiment 1, we found that size discrimination was more precise when the object was perceived to be closer to the observers. Experiments 2a and 2b extended this finding to a visual property orthogonal to depth information, by showing superior orientation discrimination for "close" objects. Finally, Experiment 3 demonstrated that the close advantage also occurs when performing high-level perceptual tasks such as face perception. Taken together, our results provide novel evidence that the perceived depth of an object, as defined by pictorial cues, modulates the precision of visual processing for close objects.

Cue vetoing in depth estimation: Physical and virtual stimuli.

Motion parallax and binocular disparity contribute to the perceived depth of three-dimensional (3D) objects. However, depth is often misperceived, even when both cues are available. This may be due in part to conflicts with unmodelled cues endemic to computerized displays. Here we evaluated the impact of display-based cue conflicts on depth cue integration by comparing perceived depth for physical and virtual objects. Truncated square pyramids were rendered using Blender and 3D printed. We assessed perceived depth using a discrimination task with motion parallax, binocular disparity, and their combination. Physical stimuli were presented with precise control over position and lighting. Virtual stimuli were viewed using a head-mounted display. To generate motion parallax, observers made lateral head movements using a chin rest on a motion platform. Observers indicated if the width of the front face appeared greater or less than the distance between this surface and the base. We found that accuracy was similar for virtual and physical pyramids. All estimates were more precise when depth was defined by binocular disparity than motion parallax. Our probabilistic model shows that a linear combination model does not adequately describe performance in either physical or virtual conditions. While there was inter-observer variability in weights, performance in all conditions was best predicted by a veto model that excludes the less reliable depth cue, in this case motion parallax.

The relationship between reflex eye realignment and the percept of single vision in young children.

Effective binocular vision is dependent on both motor and perceptual function. Young children undergo development of both components while interacting with their dynamic three-dimensional environment. When this development fails, eye misalignment and double vision may result. We compared the range of image disparities over which young children display reflex motor realignment of their eyes with the range over which they report a single versus double percept. In response to step changes in the disparity of a 2.2° wide stimulus, 5-year-olds generated an adult-like reflex vergence velocity tuning function peaking at 2° of disparity, with a mean latency of 210 ms. On average, they reported double vision for stimulus disparities of 3° and larger, compared to 1° in adult reports. Three-year-olds also generated reflex vergence tuning functions peaking at approximately 2° of disparity, but their percepts could not be assessed. These data suggest that, by age 5, reflex eye realignment responses and percepts driven by these brief stimuli are tightly coordinated in space and time to permit robust binocular function around the point of fixation. Importantly, the plastic neural processes maintaining this tight coordination during growth control the stability of visual information driving learning during childhood.

Surface slant impairs disparity discontinuity discrimination.

Binocular disparity signals are highly informative about the three-dimensional structure of visual scenes, including aiding the detection of depth discontinuities between surfaces. Here, we examine factors affecting sensitivity to such surface discontinuities. Participants were presented with random dot stereograms depicting two planar surfaces slanted in opposite directions and were asked to judge the sign of the depth discontinuity created where those surfaces met. Although the judgement was focussed on the adjacent edges, the precision of depth discontinuity discrimination depended upon the slant of the two surfaces: increasing surface slants to ±60° increased discontinuity discrimination thresholds by, on average, a factor of 5. Control experiments examining discontinuity discrimination across surfaces with identical slants showed either biases in discontinuity judgements or reduced threshold elevation. These results suggest that sensitivity to depth discontinuities is affected by processing limitations in both local absolute disparity measurement mechanisms and mechanisms selective for disparity differences. As further evidence in support of this conclusion, we show that our results are well-described by a model of discontinuity discrimination based on the encoding of local differences in relative disparity.

Stereoscopic depth constancy from a different direction.

To calibrate stereoscopic depth from disparity our visual system must compensate for an object's egocentric location. Ideally, the perceived three-dimensional shape and size of objects in visual space should be invariant with their location such that rigid objects have a consistent identity and shape. These percepts should be accurate enough to support both perceptual judgments and visually-guided interaction. This theoretical note reviews the relationship of stereoscopic depth constancy to the geometry of stereoscopic space and seemingly esoteric concepts like the horopter. We argue that to encompass the full scope of stereoscopic depth constancy, researchers need to consider not just distance but also direction, that is 3D egocentric location in space. Judgements of surface orientation need to take into account the shape of the horopter and the computation of metric depth (when tasks depend on it) must compensate for direction as well as distance to calibrate disparities. We show that the concept of the horopter underlies these considerations and that the relationship between depth constancy and the horopter should be more explicit in the literature.

Contributions of Stereopsis and Aviation Experience to Simulated Rotary Wing Altitude Estimation.

OBJECTIVE: We examined the contribution of binocular vision and experience to performance on a simulated helicopter flight task. BACKGROUND: Although there is a long history of research on the role of binocular vision and stereopsis in aviation, there is no consensus on its operational relevance. This work addresses this using a naturalistic task in a virtual environment. METHOD: Four high-resolution stereoscopic terrain types were viewed monocularly and binocularly. In separate experiments, we evaluated performance of undergraduate students and military aircrew on a simulated low hover altitude judgment task. Observers were asked to judge the distance between a virtual helicopter skid and the ground plane. RESULTS: Our results show that for both groups, altitude judgments are more accurate in the binocular viewing condition than in the monocular condition. However, in the monocular condition, aircrew were more accurate than undergraduate observers in estimating height of the skid above the ground. CONCLUSION: At simulated altitudes of 5 ft (1.5 m) or less, binocular vision provides a significant advantage for estimation of the depth separation between the landing skid and the ground, regardless of relevant operational experience. However, when binocular cues are unavailable aircrew outperform undergraduate observers, a result that likely reflects the impact of training on the ability to interpret monocular depth cues.

The impact of retinal motion on stereoacuity for physical targets.

In a series of studies using physical targets, we examined the effect of lateral retinal motion on stereoscopic depth discrimination thresholds. We briefly presented thin vertical lines, along with a fixation marker, at speeds ranging from 0 to 16 deg·s-1. Previous investigations of the effect of retinal motion on stereoacuity consistently show that there is little impact of retinal motion up to 2 deg·s-1, however, thresholds appear to rise steeply at higher velocities (greater than 3 deg·s-1). These prior experiments used computerized displays to generate their stimuli. In contrast, with our physical targets we find that stereoacuity is stable up to 16 deg·s-1, even in the presence of appreciable smearing due to visual persistence. We show that this discrepancy cannot be explained by differences in viewing time, prevalence of motion smear or by high frequency flicker due to display updates. We conclude that under natural viewing conditions observers are able to make depth discrimination judgements using binocular disparity signals that are rapidly acquired at stimulus onset.

Monovision: Consequences for depth perception from large disparities.

Recent studies have confirmed that monovision treatment degrades stereopsis but it is not clear if these effects are limited to fine disparity processing, or how they are affected by viewing distance or age. Given the link between stereopsis and postural stability, it is important that we have full understanding of the impact of monovision on binocular function. In this study we assessed the short-term effects of optically induced monovision on a depth-discrimination task for young and older (presbyopic) adults. In separate sessions, the upper limits of stereopsis were assessed with participants' best optical correction and with monovision (-1D and +1D lenses in front of the dominant and non-dominant eyes respectively), at both near (62 cm) and far (300 cm) viewing distances. Monovision viewing resulted in significant reductions in the upper limit of stereopsis or more generally in discrimination performance at large disparities, in both age groups at a viewing distance of 300 cm. Dynamic photorefraction performed on a sample of four young observers revealed that they tended to accommodate to minimize blur in one eye at the expense of blur in the other. Older participants would have experienced roughly equivalent blur in the two eyes. Despite this difference, both groups displayed similar detrimental effects of monovision. In addition, we find that discrimination accuracy was worse with monovision at the 3 m viewing distance which involves fixation distances that are typical during walking. These data suggest that stability during locomotion may be compromised, a factor that is of concern for our older participants.

Optimal combination of illusory and luminance-defined 3-D surfaces: A role for ambiguity.

The shape of the illusory surface in stereoscopic Kanizsa figures is determined by the interpolation of depth from the luminance edges of adjacent inducing elements. Despite ambiguity in the position of illusory boundaries, observers reliably perceive a coherent three-dimensional (3-D) surface. However, this ambiguity may contribute additional uncertainty to the depth percept beyond what is expected from measurement noise alone. We evaluated the intrinsic ambiguity of illusory boundaries by using a cue-combination paradigm to measure the reliability of depth percepts elicited by stereoscopic illusory surfaces. We assessed the accuracy and precision of depth percepts using 3-D Kanizsa figures relative to luminance-defined surfaces. The location of the surface peak was defined by illusory boundaries, luminance-defined edges, or both. Accuracy and precision were assessed using a depth-discrimination paradigm. A maximum likelihood linear cue combination model was used to evaluate the relative contribution of illusory and luminance-defined signals to the perceived depth of the combined surface. Our analysis showed that the standard deviation of depth estimates was consistent with an optimal cue combination model, but the points of subjective equality indicated that observers consistently underweighted the contribution of illusory boundaries. This systematic underweighting may reflect a combination rule that attributes additional intrinsic ambiguity to the location of the illusory boundary. Although previous studies show that illusory and luminance-defined contours share many perceptual similarities, our model suggests that ambiguity plays a larger role in the perceptual representation of illusory contours than of luminance-defined contours.

Disparity configuration influences depth discrimination in naïve adults, but not in children.

We report a series of experiments in which we assess depth discrimination performance in adults and children using a disparity-balanced target configuration to avoid the effects of anticipatory vergence eye movements. In our first study we found that children outperformed adults by a substantial margin, and the adults were consistently near chance. This was surprising given that we initially tested naïve adults to provide a benchmark for the children's data, and all observers met the criterion for stereoacuity. In subsequent experiments we recruited groups of inexperienced adult observers and assessed the role of a wide range of spatial and temporal factors in this apparent deficit. We found that the adult performance remained poor in spite of changes to the stimulus layout, exposure duration, and spatial scale. The only manipulations that improved performance were those that limited the binocular disparity to a single sign. We conclude that these data reflect a form of involuntary disparity pooling that makes it difficult for naïve observers to judge depth from disparity from multiple targets. The absence of this effect in children likely reflects the late maturation of global processes and depth cue integration.

Depth magnitude from stereopsis: Assessment techniques and the role of experience.

Investigations of the relationship between binocular disparity and suprathreshold depth magnitude percepts have used a variety of tasks, stimuli, and methods. Collectively, the results confirm that depth percepts increase with increasing disparity, but there are large differences in how well the estimates correspond to geometric predictions. To evaluate the source of these differences, we assessed depth magnitude percepts for simple stereoscopic stimuli, using both intra- and cross-modal estimation methods, and a large range of test disparities for both experienced and inexperienced observers. Our results confirm that there is a proportional relationship between perceived depth and binocular disparity; this relationship is not impacted by the measurement method. However, observers with minimal prior experience showed strong systematic biases in depth estimation, which resulted in large overestimates at small disparities and substantial underestimates at large disparities. By comparison, experienced observers' depth judgements were much closer to geometric predictions. In subsequent studies we show that unpracticed observers' depth estimates are improved by removing conflicting depth cues, and the observed biases are eliminated when they view physical targets. We conclude that differences in the depth magnitude estimates as a function of disparity in the existing literature are likely due to observers' experience with stereoscopic display systems in which binocular disparity is manipulated while other depth cues are held constant.

Size matters: Perceived depth magnitude varies with stimulus height.

Both the upper and lower disparity limits for stereopsis vary with the size of the targets. Recently, Tsirlin, Wilcox, and Allison (2012) suggested that perceived depth magnitude from stereopsis might also depend on the vertical extent of a stimulus. To test this hypothesis we compared apparent depth in small discs to depth in long bars with equivalent width and disparity. We used three estimation techniques: a virtual ruler, a touch-sensor (for haptic estimates) and a disparity probe. We found that depth estimates were significantly larger for the bar stimuli than for the disc stimuli for all methods of estimation and different configurations. In a second experiment, we measured perceived depth as a function of the height of the bar and the radius of the disc. Perceived depth increased with increasing bar height and disc radius suggesting that disparity is integrated along the vertical edges. We discuss size-disparity correlation and inter-neural excitatory connections as potential mechanisms that could account for these results.

Perceived three-dimensional shape toggles perceived glow.

Most surfaces reflect light from external sources, but others emit light: they glow. Glowing surfaces are often a sign of an important feature of the environment, such as a heat source or a bioluminescent life form, but we know little about how the human visual system identifies them. Previous work has shown that luminance and luminance gradients are important in glow perception [1,2]. While a link between glow and shape has been suggested in the literature [3], there has been no systematic investigation of this relationship. Here we show that perceived three-dimensional shape plays a decisive role in glow perception; vivid percepts of glow can be toggled on and off, simply by changing cues to three-dimensional shape while holding other image features constant.

Perceptual grouping via binocular disparity: The impact of stereoscopic good continuation.

Stereoscopic contextual effects are widely reported but are generally discussed in terms of 2-D Gestalt grouping principles, e.g., good continuation or closure. We propose that there are disparity-based grouping operations that are separable from 2-D grouping and instead depend on the distribution of binocular disparity information. Two experiments assess the impact of perceptual grouping via good disparity continuation. First, perceived depth magnitude is reduced for a multidot contour with a smooth disparity gradient compared to the end points in isolation. This reduction is eliminated when disparity jitter is introduced to the intermediate dots. Second, observers showed more efficient visual search for the continuous contour versus the discontinuous version. Therefore, when there is spatial support for interpretation of a slanted object, quantitative depth is reduced, but is rapidly detected in visual search. These results reflect the operation of disparity-based grouping, extending the 2-D principle of good continuation into the third dimension.